Work carried out while at the University of Maryland.

Neural Information Processing Systems http://nips.cc/

We present a new deep multi-state Dynamic Recurrent Neural Network (DRNN) architecture for Brain Machine Interface (BMI) applications. Our DRNN is used to predict Cartesian representation of a computer cursor movement kinematics from open-loop neural data recorded from the posterior parietal cortex (PPC) of a human subject in a BMI system. We design the algorithm to achieve a reasonable trade-off between performance and robustness, and we constrain memory usage in favor of future hardware implementation. We feed the predictions of the network back to the input to improve prediction performance and robustness. We apply a scheduled sampling approach to the model in order to solve a statistical distribution mismatch between the ground truth and predictions. Additionally, we configure a small DRNN to operate with a short history of input, reducing the required buffering of input data and number of memory accesses. This configuration lowers the expected power consumption in a neural network accelerator. Operating on wavelet-based neural features, we show that the average performance of DRNN surpasses other state-of-the-art methods in the literature on both single-and multi-day data recorded over 43 days. Results show that multi-state DRNN has the potential to model the nonlinear relationships between the neural data and kinematics for robust BMIs.

Neural Information Processing Systems http://nips.cc/

Speech synthesis from intracranial EEG (iEEG) signals offers a promising avenue for restoring communication in individuals with severe speech impairments. However, achieving intelligible and natural speech remains challenging due to limitations in feature representation, prosody modeling, and phase reconstruction. We introduce MiSTR, a deep-learning framework that integrates: 1) Wavelet-based feature extraction to capture fine-grained temporal, spectral, and neurophysiological representations of iEEG signals, 2) A Transformer-based decoder for prosody-aware spectrogram prediction, and 3) A neural phase vocoder enforcing harmonic consistency via adaptive spectral correction. Evaluated on a public iEEG dataset, MiSTR achieves state-of-the-art speech intelligibility, with a mean Pearson correlation of 0.91 between reconstructed and original Mel spectrograms, improving over existing neural speech synthesis baselines.

In this paper, the authors present a multi-state Dynamic Recurrent Neural Network architecture and training framework for Brain Machine Interface (BMI), including incorporating scheduled sampling and testing diverse neural features as input. The authors robustly analyze this model in comparison to other prior modeling frameworks on human posterior parietal cortical activity (PPC). This paper is of an impressive quality, containing rigorous and methodical analyses showing clear and significant improvements of their model. The authors compare to twelve baseline models and investigate many aspects of the modeling framework, including single-day vs multi-day performance, generalization of single-day training to other days, the reliance on amount of training data, the optimal preprocessing of neural feature inputs, and generalization of the models over time with different styles of retraining. The paper was very well-written, with most choices and details clearly explained.

This paper presents a deep recurrent network for decoding neural signals from the brain of a human participant for the control of a computer cursor. All reviewers thought this was an important problem and appreciated the large-scale comparison against other decoders on a pre-recorded dataset. Reviewer 1 thought the paper was of impressive quality and appreciated the experimental rigor and many aspects that were empirically evaluated. They also thought the paper was well written, but asked for more clarification regarding novelty. Reviewer 2 acknowledged the good results, but questioned the nature of the hardware problem.